Event logging has been known since the early days of sailing. Today in computer based systems and applications the idea of logging has taken new meaning. The recording of event occurrence information is the act of logging. The data captured is placed in storage devices and later processed in a variety of ways. The processing typically involves problem determination, recovery or statistical analysis to derive information related to system as well as application usage or behaviour.
Some computer systems include logging services to maintain system specific information. Such information may be stored on a local device for later use. Cases of remote logging are also known wherein the log information is transmitted over a network and stored at a location remote from the source of the logged event. Remote logging requires the knowledge of host system information to keep track of the source.
Applications which operate on top of system operating systems also create events and record the occurrence of such events. These log files are created and maintained for the purpose of monitoring application activity and status of components or other objects and activities important to the operation of the application and the care of the data created. Application log files tend to be application specific in content and format. These log files as in the case with system log files tend to be created and maintained in close proximity to the source. When such log files are remote they have the same requirements as do the system based log files, that is the identification of the source of the information which includes some form of network addressing.
Even when log files are collected from remote locations the collected log files are maintained in the same form as they were created. This creates islands of log data wherein each island is tailored to a specific kind or format of log. Interface requirements of each log type are known and required to input and extract information from the respective log files.
The uniqueness of the log file across applications and systems causes operational concerns as system and application administrators cope with the varying requirements. The uniqueness also inhibits the data sharing across log types or formats. Problems requiring data from multiple log sources typically requires the involvement of multiple people each having skill in interpreting and working with particular logs or log formats. For example an application problem may be resolved after investigating the system, application and network logs; each involving a separate person and set of tools.
As the data formats of the various logs differ so have the tools used to maintain and report on log data. Additional tools require more time and money to acquire and maintain as well as get training.